Recently, as multimedia communication services become popularized, teleconferencing systems that allow a participant to have a conversation with a counterpart, located at a remote place, face to face have been widely used. Such a teleconferencing system allows a plurality of persons to participate in a conference together, a plurality of regions to be simultaneously connected, and conference information, such as the audio or video of opposite parties, to be transmitted in real time over a network.
Generally, a teleconferencing system is configured such that a camera for capturing a user is installed over, under, or besides a screen, and the line of sight of the user is directed at the screen, so that a captured image of the user shows that the direction of the line of sight of the user does not face straight ahead.
When having a conversation, eye contact is a very important factor in engaging others in the conversation or gaining the trust of the counterpart, but a teleconferencing system that is generally used is problematic in that since the direction of the line of sight displayed on a screen does not face straight ahead, as described above, it is impossible to keep eye contact with the counterpart during the conversation. The problem of it being impossible to keep eye contact when using a teleconferencing system has acted as a fatal limitation in the actualization of teleconferencing systems, and it is one of the main contributing factors to the awkwardness that many users experience when participating in a teleconference conversation.
U.S. Pat. No. 6,771,303 presents technology for allowing participants in a video conference to align their lines of sight with one another by installing two cameras over and under a monitor upon initiating a video conference, calculating the depth information of the video conference participants, detecting the positions of heads of the participants, and generating light-of-sight images suitable for the current locations of the participants. U.S. Pat. No. 6,806,898 presents technology for allowing a current participant in a teleconference to keep eye contact with a counterpart involved in the teleconference by analyzing the position and direction of the head of the current teleconference participant via a camera installed over a monitor and by synthesizing the pupils of the eyes so that the eyes of the current participant look straight ahead.
Further, Korean Patent Application Publication No. 1997-0057779 discloses technology for generating three-dimensional (3D) images to be displayed to individual conference participants using three cameras and displaying the 3D images in three dimensions. Korean Patent Application Publication No. 2010-0028725 discloses technology for enabling only a light-receiving unit to be separately movable in an eye-contact camera, thus adaptively making eye contact even if a counterpart is not displayed at the center of a monitor.
However, such conventional eye contact-supporting technologies for a teleconferencing system merely disclose simple image acquisition via simple movement in which only a fixed camera location or the location of the counterpart on the screen is taken into consideration so as to acquire eye contact images, and does not present detailed technology related to factors that must be taken into consideration to acquire an optimal eye contact image and related to which scheme is to be used to manipulate the camera in consideration of the factors. Further, most conventional technologies adopt a technique for generating depth information by calculating real-time depth, so that there is a problem in that the accuracy of depth information is not guaranteed, thus making it difficult to generate natural images. Furthermore, the conventional technologies are technologies limited to a scheme in which participants conduct a conference while simply viewing images, and do not describe real-time interaction for more realistic teleconferencing.
Therefore, there is urgently required new 3D teleconferencing technology that provides optimal eye contact images by presenting factors to be taken into consideration so as to generate realistic eye contact images, suitably utilizing these factors, and manipulating cameras, and that enables real-time interaction for a more realistic teleconference and is capable of generating more accurate and realistic images using depth information acquired by depth cameras.